Image inspection apparatus, image inspection method, image inspection program, computer-readable recording medium and recording device

ABSTRACT

An image inspection apparatus includes: an imaging section for capturing an image of a workpiece from a certain direction; an illumination section for illuminating the workpiece from different directions at least three times; an illumination controlling section for sequentially turning on the illumination sections one by one; an imaging generating section for driving the imaging section to generate a plurality of images; a normal vector calculating section for calculating a normal vector with respect to the surface of the workpiece at each of pixels by use of a pixel value of each of pixels having a corresponding relation among the plurality of images; and a contour image generating section for performing differential processing in an X-direction and a Y-direction on the calculated normal vector at each of the pixels, to generate a contour image that shows a contour of inclination of the surface of the workpiece.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims foreign priority based on Japanese PatentApplication No. 2014-119135, filed Jun. 9, 2014, the contents of whichis incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image inspection apparatus, an imageinspection method, an image inspection program, and a computer-readablerecording medium or a recording device, each using a photometric stereomethod.

2. Description of Related Art

There has been used an image inspection apparatus for performinginspection of the presence or absence of a flaw on the surface of aworkpiece (inspection target or subject), an external shape thereof, andreading of a printed character thereon. Such an image inspectionapparatus performs necessary illumination on a workpiece, captures itsimage, and performs image processing such as edge detection on theobtained image data, to perform determination such asdefective/non-defective determination based on a result of theprocessing.

However, in such an image inspection apparatus, there has been a casewhere the viewing easiness varies by a type of illumination or adirection of applying illumination, depending on a type of theworkpiece. For this reason, performing appropriate inspection on such aworkpiece has required sufficient information and experiences.

Further, in the conventional image inspection apparatus, erroneousinspection is apt to occur due to a small change in illuminationcondition, installation condition or the like, and stably performinginspection is difficult, which has been problematic. Further, in visualinspection of the workpiece, although both information on a shape of theworkpiece such as a flaw and an edge, i.e., three-dimensionalinformation, and planar information such as a printed character and astain are inspection targets, those information cannot be detected wellin some cases as a result of interference of the information, which hasalso been problematic.

As a technique for solving such problems, there is known an imageprocessing apparatus for acquiring height information by a photometricstereo method (e.g., see Unexamined Japanese Patent Publication No.2007-206797). Here, the photometric stereo method is one of techniquesfor three-dimensional measurement where an image is captured byillumination from a plurality of different illumination directions tofind a normal vector of a workpiece from shadow information on theimage. An image processing apparatus using such a photometric stereomethod creates, from a normal vector (corresponding to an inclinationimage), an image with components on an X-axis and a Y-axis replaced by aluminance value or a reflectance image (corresponding to an albedoimage), and applies the image to image inspection. Here, in order toaccurately perform three-dimensional measurement by the photometricstereo method, consideration has been made mainly on a method forinstallation of an illumination device and a method for irradiation withillumination light.

However, in the image inspection by the photometric stereo method, sinceaccurate information of a positional relation between a light source anda workpiece is required for calculating an accurate normal vector, forexample when the surface on which the workpiece is placed is inclinedfrom a horizontal plane or illumination is inclined, a number of majorchanges occur to cause deterioration in accuracy, which has beenproblematic.

Further, calculating an accurate normal vector requires a great numberof images obtained by illumination in different directions, to takeenormous amount of processing time, which has also been problematic. Onthe other hand, even when a height dimension of the workpiece cannotnecessarily be accurately detected, it may be sufficient for use ininspection in many cases.

SUMMARY OF THE INVENTION

The present invention has been made in view of such problems, and aprincipal object is to provide an image inspection apparatus, an imageinspection method, an image inspection program, and a computer-readablerecording medium or a recording device, each of which allows easierinspection of a flaw and a printed character of a workpiece whilekeeping sufficient accuracy in a practical use by a photometric stereomethod.

An image inspection apparatus according to one embodiment of theinvention is an image inspection apparatus for performing visualinspection of a workpiece. The apparatus may include: three or moreillumination sections for illuminating a workpiece from mutuallydifferent illumination directions; an illumination controlling sectionfor turning on the three or more illumination sections one by one in aturning-on order; an imaging section for capturing an image of theworkpiece from a certain direction at illumination timing for turning oneach of the illumination sections by the illumination controllingsection, to capture a plurality of partial illumination images withdifferent illumination directions; a normal vector calculating sectionfor calculating a normal vector with respect to the surface of theworkpiece at each of pixels based on a photometric stereo method by useof a pixel value of each of pixels having a corresponding relation amongthe plurality of partial illumination images captured by the imagingsection; and a contour image generating section for performingdifferential processing in an X-direction and a Y-direction on thenormal vector at each of the pixels calculated by the normal vectorcalculating section, to generate a contour image that shows a contour ofinclination of the surface of the workpiece. With the aboveconfiguration, differential processing is performed on a normal vectorto generate a contour image that shows a contour of inclination of thesurface of the workpiece, and this contour image is applied as aninspection target image, enabling image inspection that is hardlyinfluenced by inclination of the workpiece or illumination, and is morerobust to an environment than height measurement using the conventionalphotometric stereo.

Further, in an image inspection apparatus according to anotherembodiment of the invention, the contour image generating section maycreate a plurality of reduced inclination images with a differentreduction ratio from the calculated normal vector at each of the pixels,perform differential processing in the X-direction and the Y-directionon each of the reduced inclination images, perform weighting such that areduced contour image with a predetermined reduction ratio is reflectedto the obtained reduced contour image to enlarge the image to anoriginal size, and add all the enlarged reduced contour images to form acontour extraction image.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the weighting may be performed by preparinga previously decided weighting set, and applying the weighting set tothe reduced contour image, to proportionally divide an adoption ratio ofthe reduced contour image with each of the reduction ratios.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the weighting set may include a set thatmakes large an adoption ratio of a reduced contour image by which acontour extraction image with clear roughness of the surface of theworkpiece is obtained.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the weighting set may include a set thatmakes large an adoption ratio of a reduced contour image by which acontour extraction image suitable for OCR is obtained.

Moreover, an image inspection apparatus according to still anotherembodiment of the invention may further include: a texture extractionimage generating section for calculating, from a normal vector at eachof the pixels which exists in number corresponding to the number oftimes of illumination performed by the illumination sections and iscalculated by the normal vector calculating section, albedos of each ofthe pixels in the same number as the number of the normal vectors, togenerate from the albedos a texture extraction image that shows a designobtained by removing an inclined state of the surface of the workpiece,and the contour image generating section and the texture extractionimage generating section can be switchable.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the texture extraction image generatingsection may sort values of the albedos of each of the pixels in the samenumber as the number of the normal vectors, and employ, as the textureextraction image, an image formed by adopting a pixel value in aspecific order from the top.

An image inspection apparatus according to still another embodiment ofthe invention may further include: an inspection region specifyingsection for specifying a position of an inspection region to become aninspection target with respect to the generated contour image; an imageprocessing section for performing image processing for detecting a flawwithin the inspection region specified by the inspection regionspecifying section; and a determination section for determining thepresence or absence of a flaw on the surface of the workpiece from aresult of the processing by the image processing section.

Moreover, an image inspection apparatus according to still anotherembodiment of the invention may include: three or more illuminationsections for illuminating a workpiece from mutually differentillumination directions; an illumination controlling section for turningon the three or more illumination sections one by one in a turning-onorder; an imaging section for capturing an image of the workpiece from acertain direction at illumination timing for turning on each of theillumination sections by the illumination controlling section, tocapture a plurality of partial illumination images with differentillumination directions; a normal vector calculating section forcalculating a normal vector with respect to the surface of the workpieceat each of pixels by use of a pixel value of each of pixels having acorresponding relation among the plurality of partial illuminationimages captured by the imaging section; and a texture extraction imagegenerating section for calculating, from a calculated normal vector ateach of the pixels which exists in number corresponding to the number oftimes of illumination performed by the illumination sections, albedos ofeach of the pixels in the same number as the number of the normalvectors, to generate from the albedos a texture extraction image thatshows a design obtained by removing an inclined state of the surface ofthe workpiece. The texture extraction image generating section may sortvalues of the albedos of each of the pixels in the same number as thenumber of the normal vectors, and employ, as the texture extractionimage, an image formed by adopting a pixel value in a specific orderfrom the top.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the imaging section and each of theillumination sections may be independent separate members, and may bearranged at arbitrary positions. With the above configuration, theillumination sections and the imaging section can be arranged inaccordance with an actual workpiece and installation environment,enabling highly flexible installation with an enhanced degree of freedomin arrangement.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, four illumination sections can be provided.

Further, in an image inspection apparatus according to still anotherembodiment of the invention, the three or more illumination sections aremade up of a plurality of light-emitting elements arranged in an annularshape, and the illumination controlling section takes a predeterminednumber of adjacent light-emitting elements as a first illuminationblock, simultaneously turns on the light-emitting elements in the firstillumination block, and turns off the other light-emitting elements, tomake a first illumination section function as first illumination from afirst illumination direction, performs control so as to turn on a secondillumination block, which is made up of a predetermined number oflight-emitting elements and adjacent to the first illumination block, toconstitute the second illumination block for performing illuminationfrom a second illumination direction different from the firstillumination direction, and performs control so as to turn on a thirdillumination block, which is made up of a predetermined number oflight-emitting elements and adjacent to the second illumination block,to constitute the third illumination block for performing illuminationfrom a third illumination direction different from the firstillumination block and the second illumination direction. With the aboveconfiguration, it is possible to divided illumination into three or moreillumination blocks that function as three or more illumination sectionsby means of one illumination unit annularly arranged with a plurality oflight-emitting elements, so as to further simplify the configuration ofthe illumination section.

An image inspection method according to one embodiment of the inventionis an inspection method for capturing an image of a workpiece to performvisual inspection. The method includes the steps of illuminating aworkpiece from three or more mutually different illumination directionsby illumination sections, and capturing one partial illumination imagewith respect to each of the illumination directions by use of a commonimaging section whose imaging direction and relative position with theillumination sections are adjusted in advance, to acquire a plurality ofpartial illumination images with different illumination directions;calculating a normal vector with respect to the surface of the workpieceat each of pixels by a normal vector calculating section by use of apixel value of each of pixels having a corresponding relation among theplurality of partial illumination images with different illuminationdirections; performing differential processing in an X-direction and aY-direction on the calculated normal vector at each of the pixels, togenerate a contour image that shows a contour of inclination of thesurface of the workpiece by a contour image generating section; andperforming visual inspection of the workpiece by use of the contourimage.

An image inspection method according to another embodiment of theinvention is an image inspection program for capturing an image of aworkpiece to perform visual inspection, and the program allows acomputer to realize functions of illuminating a workpiece from three ormore mutually different illumination directions by illuminationsections, and capturing one partial illumination image with respect toeach of the illumination directions by use of a common imaging sectionwhose imaging direction and relative position with the illuminationsections are adjusted in advance, to acquire a plurality of partialillumination images with different illumination directions; calculatinga normal vector with respect to the surface of the workpiece at each ofpixels by a normal vector calculating section by use of a pixel value ofeach of pixels having a corresponding relation among the plurality ofpartial illumination images with different illumination directions;performing differential processing in an X-direction and a Y-directionon the calculated normal vector at each of the pixels, to generate acontour image that shows a contour of inclination of the surface of theworkpiece by a contour image generating section; and performing visualinspection of the workpiece by use of the contour image.

Further, a computer-readable recording medium or a recording device ofthe present invention store the image inspection program. The recordingmedium includes a magnetic disk, an optical disk, a magneto-opticaldisk, a semiconductor memory, and other program-storable medium, such asa CD-ROM, a CD-R, a CD-RW, a flexible disk, a magnetic tape, an MO, aDVD-ROM, a DVD-RAM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a Blu-ray(product name), and an HD-DVD (AOD). Further, the program is distributedby downloading through a network such as the Internet, other than storedinto the above recording medium. Moreover, the recording device includesa general-purpose or special-purpose device where the program is mountedin the form of software, firmware or the like, in an executable state.Furthermore, each processing and each function included in the programmay be executed by program software that is executable by the computer,and processing of each part may be realized by predetermined hardwaresuch as a gate array (FPGA, ASIC) or in the form of program softwarebeing mixed with a partial hardware module that realizes some element ofhardware.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an image inspectionapparatus according to a first embodiment of the present invention;

FIG. 2 is a schematic plan view showing a positional relation between animaging section and each of illumination sections of the imageinspection apparatus of FIG. 1;

FIG. 3 is a schematic side view showing the positional relation betweenthe imaging section and each of the illumination sections of the imageinspection apparatus of FIG. 1;

FIG. 4 is a schematic view showing an illumination section that realizesfour illumination blocks according to a second embodiment;

FIG. 5 is a schematic plan view showing a state where five illuminationblocks are realized by the illumination section of FIG. 4;

FIG. 6 is a schematic plan view showing illumination sections accordingto a third embodiment;

FIG. 7 is a schematic plan view showing illumination sections accordingto a fourth embodiment;

FIG. 8 is a schematic plan view showing an illumination sectionaccording to a modified example;

FIG. 9 is a schematic plan view showing an illumination sectionaccording to another modified example;

FIG. 10 is a schematic view showing a configuration of an imageinspection apparatus according to a fifth embodiment;

FIG. 11 is a schematic view showing a configuration of an imageinspection apparatus according to a sixth embodiment;

FIG. 12 is a block diagram for use in explaining a signal processingsystem of an image processing section;

FIG. 13A is a view showing a positional relation between a diffusingreflective surface S and illumination, FIG. 13B is a view showing astate where irradiation is performed with light from L1, FIG. 13C is aview showing a state where light has been applied from L2, and FIG. 13Dis a view for explaining that an orientation of the surface is estimatedfrom combination of an irradiation direction and brightness ofreflective light, each for explaining a basic principle of a photometricstereo method;

FIG. 14A is a view showing one example of an inclination imagedifferentiated in a vertical direction, FIG. 14B is a view showing oneexample of an inclination image differentiated in a horizontaldirection, and FIG. 14C is a view showing one example of a contourextraction image, each for explaining a method for generating a contourextraction image;

FIG. 15A is a view showing surface information, FIG. 15B is a diagramshowing an inclination image, FIG. 15C is a view showing a forwarddifference, and FIG. 15D is a view showing a central difference, eachfor explaining a method for calculating δ²s/δx² and δ²s/δy²;

FIG. 16A is a view showing a source image, FIG. 16B is a view showing animage subjected to processing by an average method, and FIG. 16C is aview showing an image subjected to processing by a halation removingmethod, each for explaining a method for generating a texture extractionimage;

FIG. 17 is a diagram for use in explaining angle-noise reduction;

FIG. 18 is a schematic cross section showing a state where, in the caseof an illumination-camera integrated model, it interferes with anobstacle;

FIG. 19 is a schematic view showing a state where the camera blocks partof illumination light;

FIG. 20 is a schematic sectional view showing a configuration of anillumination-camera separate model;

FIG. 21A is a view showing a case where an LWD is made short, and FIG.21B is a view showing a case where the LWD is made long, each forexplaining an advantage of the illumination-camera separate model;

FIG. 22 is a schematic view showing a state where a ring-likeillumination is arranged;

FIG. 23 is a schematic view showing a state where rotational positionsof the ring-like illumination and the camera are matched with eachother;

FIG. 24 is a schematic view showing an installation auxiliary sectionaccording to a first example;

FIG. 25 is a schematic view showing an example where the installationauxiliary section has been applied to connection between an illuminationcable and the illumination section;

FIG. 26 is a schematic view showing an installation auxiliary sectionaccording to a second example;

FIG. 27 is a schematic view showing an installation auxiliary sectionaccording to a third example;

FIG. 28 is a schematic view showing an installation auxiliary sectionaccording to a modified example;

FIG. 29 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 30 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 31 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 32 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 33 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 34 is a schematic view showing an installation auxiliary sectionaccording to another modified example;

FIG. 35 is a flowchart showing an image inspection method according toan example; and

FIG. 36 is a flowchart showing an image inspection method according to amodified example.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the drawings. The embodiments shown below are for embodyingtechnical ideas of the present invention, and the present invention isnot limited to the following. Further, the present specification doesnot limit members shown in the claims to members of the embodiments.Especially, dimensions, materials, shapes, relative disposition and thelike of constituent components described in the embodiments are notintended to restrict the scope of the present invention thereto, but aremere explanatory examples, unless particularly specifically described.It is to be noted that sizes, positional relations and the like ofmembers shown in each drawing may be exaggerated for clarifying adescription. Further, in the following description, the same name orsymbol denotes the same member or members of the same quality, and adetailed description thereof will be omitted as appropriate. Moreover,each element constituting the present invention may have a mode where aplurality of elements are made up of the same member and the one membermay serve as the plurality of elements, or conversely, a function of onemember can be shared and realized by a plurality of members.

(1. Configuration of Image Inspection Apparatus 1)

FIG. 1 shows a block diagram of an image inspection apparatus accordingto a first embodiment of the present invention. The image inspectionapparatus 1 shown in this drawing includes: an imaging section 11 thatcaptures an image of an inspection target (hereinafter also referred toas “workpiece WK”) from a certain direction; illumination sections forilluminating the workpiece WK from three or more different illuminationdirections; an illumination controlling section 31 for turning on eachof the illumination sections one by one in a turning-on order; and animage processing part 41 that is connected with the illuminationcontrolling section and the imaging section to control these sections.The image processing part 41 and the imaging section are connected viaan image capturing cable 12, and the image processing part 41 and theillumination controlling section 31 are connected via an illuminationcable 32. Further, the image processing part connects a display section51 and an operation section 61. Moreover, the image processing part canalso be connected with a PLC (Programmable Logic Controller), a computerand the like as external devices, according to the need.

The imaging section captures an image of the workpiece WK from a certaindirection at illumination timing for turning on each of the illuminationsections by the illumination controlling section, to capture a pluralityof partial illumination images with different illumination directions.

The image processing part realizes functions of a normal vectorcalculating section 41 a, a contour image generating section 41 b, atexture extraction image generating section 41 c, an inspection regionspecifying section 41 d, an image processing section 41 e, and adetermination section 41 f. The normal vector calculating section 41 acalculates a normal vector n with respect to the surface of theworkpiece WK at each of pixels by use of a pixel value of each of pixelshaving a corresponding relation among the plurality of partialillumination images captured by the imaging section. The contour imagegenerating section 41 b performs differential processing in anX-direction and a Y-direction on the calculated normal vector n at eachof the pixels, to generate a contour image that shows a contour ofinclination of the surface of the workpiece WK. The texture extractionimage generating section 41 c calculates, from the calculated normalvector n at each of the pixels which exists in number corresponding tothe number of times of illumination performed by the illuminationsections, albedos of each of the pixels in the same number as the numberof the normal vectors n, to generate from the albedos a textureextraction image that shows a design obtained by removing an inclinedstate of the surface of the workpiece WK. The inspection regionspecifying section 41 d specifies a position of an inspection region tobecome an inspection target with respect to the generated contour image.The image processing section 41 e performs image processing fordetecting a flaw within the specified inspection region. Thedetermination section 41 f determines the presence or absence of a flawon the surface of the workpiece WK based on the processing result.

The imaging section and the illumination section can be arranged asseparate members. This allows a layout with a high degree of freedom. Asone example shown in a schematic plan view of FIG. 2 and a schematicside view of FIG. 3, the imaging section 11 with an optical axis turnedin a vertical direction is arranged immediately above the workpiece WKplaced on a stage SG. Further, four illumination sections, namely afirst illumination section 21, a second illumination section 22, a thirdillumination section 23 and a fourth illumination section 24, arearranged at the same height in the cardinal directions of north, south,east and west of the imaging section 11. A positional relation betweenthe imaging section and each of the illumination sections having beenarranged are recorded on the image inspection apparatus. Each of theillumination sections is sequentially turned on at predeterminedillumination timing by the illumination controlling section, and animage of the workpiece is captured from a certain direction by thecommon imaging section, to acquire partial illumination images.

The configuration where the imaging section and the illumination sectionare separate members is not restrictive, and these may be integrallyconfigured via an arm or the like. In this case, since the positionalrelation between the imaging section and each of the illuminationsections is fixed in advance, an adjustment operation such as matchingof the optical axes can be made unnecessary. However, the degree offreedom will be lost.

(Imaging Section)

As for the imaging section 11, for example, an image capturing elementsuch as a CCD (Charge Coupled Device) camera or a CMOS (ComplementaryMetal Oxide Semiconductor) imager can be used. The image capturingelement performs photoelectric conversion on an image of a subject tooutput an image signal, and a signal processing block converts theoutputted image signal to a luminance signal and a color differencesignal, to output the signals to the image processing part 41 connectedby the image capturing cable 12.

(Illumination Section)

The illumination sections 21, 22, 23, 24 are arranged so as to surroundthe workpiece WK as shown in the schematic plan view of FIG. 2 such thatthe workpiece WK can be irradiated with illumination light fromdifferent illumination directions. Further, as shown in the schematicside view of FIG. 3, each of the illumination sections is arranged withthe optical axis turned obliquely below. It is preferable to match theoptical axis of the imaging section with a central axis of a plane(imaginary rotational plane) provided with the illumination sections sothat an image of the workpiece illuminated by each of the illuminationsections can be captured by the common imaging section. Further, it ispreferable to set an interval (azimuth from the central axis) betweenthe illumination sections by uniformly dividing 360° by the number ofillumination sections. Moreover, it is preferable to make a zenith angleconstant in all the illumination sections. Furthermore, it is alsopreferable to make a distance between each of the illumination sectionsand the workpiece constant. This can simplify input of information onthe azimuth and the zenith angle which are required for computing inphotometric stereo processing. Further, as described later, since anentirely turning-on image MC is captured in an entirely turning-on statewhere all the illumination sections are on, imaging can be performed inan illumination state with little unevenness just by uniformly reducingthe intensity of the entire illumination with the above configuration.

In the example of FIG. 1, the illumination sections are made up of foursections: the first illumination section 21, the second illuminationsection 22, the third illumination section 23 and the fourthillumination section 24. For each of the illumination sections, anincandescent light bulb, a fluorescent lamp or the like can be used. Inparticular, a semiconductor light-emitting element such as a lightemitting diode (LED) is preferable as having small power consumption, along life and excellent responsiveness. As shown in FIG. 1, theillumination sections are connected to an illumination dividing unit 75via the respective cables 71, 72, 73, 74, and are further connected tothe illumination controlling section 31 via a cable 76.

(Illumination Dividing Unit)

The illumination dividing unit is an interface for connecting each ofthe illumination sections and the illumination controlling section.Specifically, an illumination connector for connecting the illuminationcable extending from the illumination section is provided. In theexample of FIG. 1, four illumination connectors are provided so as toconnect the four illumination sections. Here, in order to correctlyconnect the illumination cable to the illumination connector, a mark orthe like may be provided as an installation auxiliary section (detailedlater). Correctly connecting the illumination cable of each of theillumination sections to the illumination connector of the illuminationdividing unit allows the illumination section to be turned on from acorrect direction in a correct order by the illumination controllingsection, and further, operating the imaging section in synchronizationwith each illumination timing allows the partial illumination image tobe captured. In addition, although the illumination dividing unit isprovided as a separate body from the illumination controlling section inthe example of FIG. 1, this configuration is not restrictive, and forexample, the illumination dividing unit may be incorporated into theillumination controlling section.

An illumination color of each of the illumination sections 21, 22, 23 24can also be changed in accordance with a type of the workpiece WK. Forexample, when a small flaw is to be inspected, blue illumination with ashort wavelength is preferable. When a colored workpiece is to beinspected, white illumination is preferably used so that the color ofthe illumination does not become obstructive. When oil is on theworkpiece, red illumination may be adopted for preventing an influencethereof.

Although the number of illumination sections is four in the example ofFIGS. 1 to 3, at least three illumination sections can be sufficientlyused so as to allow the workpiece WK to be illuminated from three ormore different illumination directions. When the number of illuminationsections is increased, the partial illumination images from moreillumination directions can be obtained, and hence the accuracy in imageinspection can be improved. For example, directions of northeast,northwest, southeast and southwest may be added and a total of eightillumination sections may be arranged. Further, it is preferable to setan interval (azimuth from the central axis) between the illuminationsections by uniformly dividing 360° by the number of illuminationsections. Moreover, it is preferable to make a zenith angle constant inall the illumination sections. It is to be noted that an increase innumber of images to be processed leads to an increase in processingamount, which slows the processing time. In the present embodiment, inview of balance of the processing speed, the easiness to perform thearithmetic processing, the accuracy and the like, the number ofillumination sections is set to four as described above.

Further, the illumination section can also be made up of a plurality ofannularly arranged light-emitting elements. For example, in ringillumination according to a second embodiment shown in FIG. 4, annularlyarranged light-emitting elements are divided into four illuminationblocks. Then, the illumination controlling section takes a firstillumination block as the first illumination section, a secondillumination block as the second illumination section, a thirdillumination block as the third illumination section and a fourthillumination block as the fourth illumination section, and makesillumination timing for the respective illumination blocks different,thereby allowing control in a similar manner to the case of fourseparate illumination sections existing.

Further, with this configuration, there can be obtained an advantagethat the number of illumination sections can be arbitrary changed by useof the same ring illumination. That is, when turning-on of each of thelight-emitting elements can be arbitrarily controlled by theillumination controlling section, as shown in FIG. 5, the number ofillumination blocks obtained by dividing the circumference of theannularly arrayed light-emitting elements is changed from four to five,and the illumination controlling section performs control so as tocapture each of partial illumination images by shifting the illuminationtiming for turning on each of the five illumination blocks. Thus, it ispossible to acquire partial illumination images from five illuminationdirections. Further, similarly, when the annular circumference isdivided into six or seven blocks, the illumination directions canfurther be increased. Moreover, the configuration where partialillumination images are constantly acquired by certain illuminationblocks is not restrictive, and illumination blocks may be changed ineach cycle. As thus described, by adjusting the turning-on pattern ofeach of the light-emitting elements, it is possible to virtually changethe number of illumination blocks by use of the same one ringillumination, so as to obtain a similar effect to that obtained byadjusting the number of illumination sections. In other words, it ispossible to deal with different accuracies by means of the commonhardware.

Further, other than arranging the illumination sections in the annularform, it is also possible to arrange illumination sections, each ofwhich is configured in a bar shape, in a rectangular form as a thirdembodiment as shown in a schematic plan view of FIG. 6, or it is alsopossible to arrange illumination sections in a polygonal form as afourth embodiment as shown in a schematic plan view of FIG. 7.

Alternatively, it is also possible to arrange the illumination sectionsin a flat form other than being arranged in a circular or polygonalannular form. For example, a large number of light-emitting elements arearranged in a flat form and an illumination block to be turned on ischanged, thereby allowing realization of different illuminationdirections. Specifically, as an illumination section according to amodified example shown in FIG. 8, an illumination unit 20′ obtained bysuperimposing concentrically annular rings is configured, and theillumination section is configured by the annular rings with differentradiuses as respective illumination blocks. Alternatively, as anillumination section according to another modified example shown in FIG.9, an illumination unit 20″ obtained by arraying light-emitting elementsin a dot-matrix form may be configured, and the illumination section maybe configured by illumination blocks obtained by dividing theillumination unit by a plurality of line segments passing through itscenter. As thus described, the illumination section and the illuminationdirection in the present invention are not restricted to physicallyseparated illumination, but are used in the meaning of including aconfiguration where illumination is performed by means of illuminationblocks obtained by dividing one illumination section into a plurality ofblocks.

It is to be noted that in the present example, the processing isperformed on the assumption that partial illumination light by each ofthe illumination sections is parallel light within an imaging range. Solong as the partial illumination light is parallel light, only thedirection of the illumination light (e.g., any of north, south, east andwest) is a concern, and other detailed positions, such as a coordinateposition of a light source of the illumination section, are not requiredto be considered.

(Illumination Controlling Section)

The illumination controlling section performs control so as to turn onthree or more illumination sections one by one in a turning-on order andsynthesize each of the illumination sections and the imaging sectionsuch that an image of the workpiece is captured by the imaging sectionfrom a certain direction at illumination timing for turning on each ofthe illumination sections. In other words, the illumination controllingsection synthesizes the timing for illumination by the illuminationsection with the timing for imaging by the imaging section. Further, theturning-on order in which the illumination controlling section turns oneach of the illumination sections may be such that the illuminationsections arranged to surround the workpiece are turned on in a clockwiseorder or a counterclockwise order, or in a discrete order such as analternate order or a crossing order. Whatever the order is, it ispossible to construct a normal vector image by the photometric stereomethod, by grasping an illumination direction of illumination by which apartial illumination image has been captured at each illuminationtiming.

It is to be noted that in the first embodiment of FIG. 1, theillumination controlling section 31 is provided as a separate body fromthe image processing part 41, but this configuration is not restrictive.For example, the illumination controlling section 31 may be integratedwith the image processing part 41 as in a fifth embodiment shown in FIG.10, or it may be built in an illumination section 25 as in a sixthembodiment shown in FIG. 11.

(Image Processing Part)

The image processing part 41 controls operations of the imaging section11 and the illumination sections 21, 22, 23, 24. Further, by use ofimage signals Q1 to Q4 of four partial illumination images inputted fromthe imaging section 11, the image processing part 41 generates a normalvector image (hereinafter referred to as “inclination image”) on a planeat each pixel, and creates, from the inclination image, a secondaryinclination image (hereinafter referred to as “contour extractionimage”) in the X-direction and the Y-direction and an albedo (meaning areflectance) image (hereinafter also referred to as “texture extractionimage”). Then, by use of those images, the image processing part 41performs processing for inspecting a flaw, detecting a character, or thelike. It should be noted that the processing for inspecting a flaw,detecting a character, or the like is not restricted to theconfiguration where the processing is performed in the image processingpart 41, and for example, it can be executed on the external device sidesuch as a PLC 81.

FIG. 12 shows a signal processing system 42 of the image processing part41. The signal processing system 42 is made up of a CPU 43, a memory 44,a ROM 45, a display section 51, an operation section 61 such as apointing device, the imaging section 11, the illumination controllingsection 31, and a PLC (program logic controller) 81 for executing resultoutput processing, and these are mutually connected via a bus 46 andcables 12, 32, 52, 62 and 82. It is to be noted that the ROM 45 may be aportable medium. Further, the display section 51 may be used togetherwith the operation section 61 by forming a touch panel or the like.

Based on a program stored in the ROM 45, the CPU 43 controlstransmission and reception of data among the memory 44, the ROM 45, thedisplay section 51, the operation section 61, the imaging section 11,the illumination controlling section 31 and the PLC 81, and controls thedisplay section 51, the imaging section 11 and the illuminationcontrolling section 31.

Although the image processing part 41 is assumed to be, for example, onecomputer stored with a program, but each section may be configured bycombination of a plurality of computers, or part of the sections may beconfigured of a dedicated circuit. Alternatively, the image processingpart 41 can be a dedicatedly designed member such as an ASIC.

(Determination Section)

The image processing part 41 realizes the function of the determinationsection as described above. The determination section inspects thepresence or absence of a flaw or a size of the flaw based on theobtained texture extraction image. For example, when the obtained valueis not smaller than a predetermined threshold, determination as a flawis made. Further, according to the need, the determination section canalso perform OCR based on a contour extraction image, to output arecognized character string.

(Basic Principle)

Next, by use of the above image inspection apparatus, a basic principlein performing visual inspection of the workpiece will be described whileit is compared with the technique of Unexamined Japanese PatentPublication No. 2007-206797 as the conventional technique. First, abasic principle of the technique disclosed in Unexamined Japanese PatentPublication No. 2007-206797 is that, by use the principle of thephotometric stereo method, light is applied to an unknown surface from avariety of directions and the shape of the workpiece is estimated usingdifferences in reflective light of the workpiece. The reflective lightof the workpiece is affected by an incident angle of illumination, adistance from illumination and the like, and has a property that thelight is brightest when the incident angle is 90° and the light becomesdarker as the distance from the illumination becomes longer.

With this property, a plurality of illuminations whose brightness andpositions are known are prepared and turning-on of the illumination issequentially switched, to estimate in which direction the surface isturned by use of a difference in brightness of the reflective light atthe time of irradiation with light from the illumination in eachdirection. Specifically, an X-component image obtained by replacing theX-component and the Y-component of the inclination image with luminanceof the X-component, and a Y-component image obtained by replacing theX-component and the Y-component of the inclination image with luminanceof the Y-component are to be created and applied to inspection.

However, this method has a problem of inferior robust characteristicsbecause an obtained inspection image greatly changes by slightinclination of the illumination or the installation surface of theworkpiece, or an error of input information such as an originallyinputted illumination position. For example, the method has adisadvantage that an inspection image corresponding to an actual shapeof the workpiece can not necessarily be obtained, as seen in a casewhere an image of roughness is obtained although there actually is noroughness, and a case where an image in which the center of theworkpiece is swelled due to a change in brightness is seen while acloser position of an image to illumination is normally seen morebrightly.

In contrast, a basic principle of the image inspection techniqueaccording to the present embodiment is as follows. Although a primaryinclination image is generated by the photometric stereo method atfirst, a secondary inclination image, namely a contour extraction imageis created by performing differential processing on the generatedprimary inclination image in the X-direction and the Y-direction, andinspection of a flaw and the like is performed with that image. Even inthe case of occurrence of the foregoing disadvantage, the influence onthe secondary inclination image is small, and by setting a place with alarge change in surface inclination to have a dark tone and setting aplace with a small change in surface inclination to have a bright tone,the secondary inclination image becomes a preferable image forextracting a flaw, a contour and the like where the inclination of thesurface of the workpiece greatly changes.

Further, in the technique disclosed in Unexamined Japanese PatentPublication No. 2007-206797, halation occurs in the reflectance image(corresponding to the texture extraction image) generated by thephotometric stereo method, and it may be difficult to detect acharacter, and the like. In contrast, in the image inspection techniqueaccording to the present embodiment, by use of the basic principle thathalation basically does not occur in the same place unless twoilluminations are used, for example, the third largest pixel value offour pixel values is adopted at each pixel, to remove an influence ofhalation.

Additionally, in the image processing apparatus disclosed in UnexaminedJapanese Patent Publication No. 2007-206797, the camera and the lightsource for illumination are integrally configured, but with thisconfiguration, the camera and the light source increase in size, and atthe time of installation, those are restricted in size, which has beenproblematic. In contrast, in the image inspection apparatus according tothe present embodiment, the imaging section and the illumination sectioncan be made to be separate members, to allow more flexible installationin view of an arrangement space, which is also an advantage in terms ofusability.

(Basic Principle of the Photometric Stereo Method)

Here, the basic principle of the photometric stereo method will bedescribed with reference to FIGS. 13A to 13D. First, as shown in FIG.13A, there is assumed a case where an unknown diffusing reflectivesurface S and a plurality of illuminations whose brightness andpositions are known (in this example, two illuminations: a firstillumination section L1 and a second illumination section L2) arepresent. For example, as shown in FIG. 13B, when irradiation isperformed with light from the first illumination section L1, diffusingreflective light on the surface of the diffusing reflective surface S isdecided only by: (1) brightness of the illumination (known); (2) anorientation of the illumination (known); (3) an orientation of thesurface of the workpiece WK (normal vector n); and (4) a parameter of analbedo of the surface of the workpiece WK.

Therefore, as shown in FIG. 13B and FIG. 13C, each of partialillumination images, obtained from diffusing reflective light at thetime of projection of illumination light from a plurality of differentillumination directions, specifically three or more illuminationdirections, is captured by the imaging section. Then, as shown in FIG.13D, three or more partial illumination images are employed as inputimages, thereby allowing calculation of: (3) an orientation of thesurface of the workpiece WK (normal vector n); and (4) an albedo of thesurface of the workpiece WK, which are unknown, from the followingrelational expression.I=ρLSn

where ρ is an albedo, L is brightness of the illumination, S is a matrixin the illumination direction, n is a normal vector of the surface, andI is a tone value of the image.

From the above expression, when the number of illumination sections isthree, the following expression is given.

$\begin{matrix}{\begin{pmatrix}I_{1} \\I_{2} \\I_{3}\end{pmatrix} = {\rho\;{L\begin{pmatrix}s_{11} & s_{12} & s_{13} \\s_{21} & s_{22} & s_{23} \\s_{31} & s_{32} & s_{33}\end{pmatrix}}\begin{pmatrix}n_{x} \\n_{y} \\n_{z}\end{pmatrix}}} & \left\lbrack {{Mathematical}\mspace{14mu}{Expression}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Further, when the number of illumination sections is four, the followingexpression is given.

$\begin{matrix}{{\begin{pmatrix}I_{1} \\I_{2} \\I_{3} \\I_{4}\end{pmatrix} = {\rho\;{L\begin{pmatrix}s_{11} & s_{12} & s_{13} \\s_{21} & s_{22} & s_{23} \\s_{31} & s_{32} & s_{33} \\s_{41} & s_{42} & s_{43}\end{pmatrix}}\begin{pmatrix}n_{x} \\n_{y} \\n_{z}\end{pmatrix}}}\left( {{Normal}\mspace{14mu}{vector}\mspace{14mu} n} \right)} & \left\lbrack {{Mathematical}\mspace{14mu}{Expression}\mspace{14mu} 2} \right\rbrack\end{matrix}$

From the above expression, the normal vector n can be expressed by thefollowing expression.n=1/ρL·S ⁺ I

In the above expression, when “S⁺: a square matrix”, “a normal inversematrix S⁺: an inverse matrix of a longitudinal matrix” is found byMoore-Penrose pseudo-inverse matrix S⁺=(S^(t)S)⁻¹S^(t).

(Albedo)

Further, the albedo ρ can be expressed by the following expression:ρ=|I|/|LSn|(2-2. Contour Extraction Image)

Next, a description will be given of a method for generating aninclination image by the photometric stereo method and obtaininginformation of the surface of the workpiece, such as a flaw and acontour, from the obtained inclination image.

(Inclination Image)

First, a method for generating the inclination image will be described.When it is assumed that the curved surface of the workpiece is S, theinclination image is given by the following expression:X-direction: δs/δx, Y-direction: δs/δy

Here, as examples of the inclination image, FIGS. 14A and 14B showexamples of using a one-yen coin as the workpiece. FIG. 14A is aY-coordinate component image in a normal direction, and FIG. 14B is anX-coordinate component image in a normal direction. Here, by use ofpartial illumination images captured from four illumination directions,an inclination image shown in FIG. 14A is obtained by performingdifferentiation in the Y-direction (a vertical direction in thedrawing), and an inclination image shown in FIG. 14B is obtained byperforming differentiation in the X-direction (a horizontal direction inthe drawing).

Here, since a flaw, a contour and the like are places where theinclination of the surface of the work piece changes, the inclinationimages are differentiated in the respective directions. The secondaryinclination image is given by the following expression:X-direction: δ² s/δx ² , Y-direction: δ² s/δy ²(Contour Extraction Image)

Thus, the portions δ²s/δx², δ²s/δy² of the inclination images in theX-direction and the Y-direction are synthesized to generate a contourextraction image including information of a contour and a flaw of theworkpiece. A contour extraction image E is given by the followingexpression.E=δ ² s/δx ²+δ² s/δy ²

In the above expression, E represents contour information, and Srepresents the curved surface of the workpiece. FIG. 14C shows anexample of the contour extraction image computed from FIGS. 14A and 14B.In the contour extraction image, a height is expressed by gradation(luminance) of the image such that a high portion is colored in whiteand a low portion is colored in black.

(Differentiation Synthesis Method)

Examples of a differentiation synthesis method that is performed ingenerating the contour extraction image include: (1) simple addition;(2) multiple resolution; and (3) square sum.

(1. Simple Addition)

Here, “(1) simple addition” is a sum of differentials of X/Y-inclinationimages at each pixel.

(2: Double Resolution)

Further, “(2) multi-resolution” is obtained by creating a plurality ofreduced inclination images, obtained by reducing the inclination imageat different reduction ratios, and finding intensity of a contour ineach of the reduced inclination images by the method of (1). Thereduction ratios are, for example, 1/1, 1/2, 1/4, 1/8, 1/16, 1/32 andthe like. The plurality of reduced contour images as thus obtained aresubjected to predetermined weighting and enlargement processing, and animage obtained by adding all the enlarged reduced contour images isregarded as a contour extraction image. Here, when the weighting ischanged, a flaw, a contour and the like each having an arbitrarythickness can be extracted.

(3. Square Sum)

Further, in “(3) square sum”, a contour extraction image is created inwhich a sum of a square of differentials of the X/Y-inclination imagesis regarded as intensity of a contour. It is to be noted that “(2)multi-resolution” is adopted in the present embodiment.

A size used for flaw determination varies depending on the user's use.For example, a depression over ten pixels may be determined as a flaw,or a depression over 100 pixels may be determined as a flaw. Further,only a steep edge may be to be extracted as an edge.

When the number of pixels of the inclination image is large, it isregarded as a large flaw in the processing. Therefore, when a large flawis to be extracted, an inclination image is reduced, intensity of acontour is found by the method of (1), and then the image is enlarged.On the other hand, when a small flaw is to be extracted, differentialsynthesis may be performed by the method of (1) without performingweighting.

That is, in the weighting, a previously decided weighting set isprepared at the time of synthesis, and reduced inclination images of allthe kinds described above are created. Then, when a large flaw is to beseen, a result from the more reduced image is weighted, and when a smallflaw is to be seen, a result from the less reduced image is weighted.

Here, the contour extraction image is obtained by adding all the reducedcontour images having been enlarged. Since a flaw is normally detectedover a plurality of frequencies, when the frequency is limited to onefrequency, for example, only a flaw detected at that limited frequencyis extracted and hence the image blurs as a whole.

(Characteristic Size)

The foregoing weighting set is formed such that, for example, aparameter named a “characteristic size” is provided, the thinnest flawcan be detected when this value is 1, and a larger flaw is detected asthis value continues to be increased. When the characteristic sizecontinues to be increased and the image comes into a state where alarger flaw is easy to detect, the roughness of the surface of theworkpiece becomes more apparent. Therefore, a predetermined thresholdmay be provided for the characteristic size, and a case where thecharacteristic size is equal to or larger than the threshold is aroughness mode, which may then be used separately from a contourextraction mode, depending on the characteristic size of the contourextraction image.

Next, a method for calculating δ²s/δx² and δ²s/δy² will be described.Examples of this calculation method include: (1) forward difference; and(2) central difference.

(1. Forward Difference)

In the forward difference, an inclination image Gh in the horizontaldirection and an inclination image Gv in the vertical direction areregarded as input, and a pixel G(x,y) at coordinates (x,y) of a contourimage E is calculated by the following expression:E(x,y)=Gh(x−1,y)−Gh(x,y)+Gv(x,y−1)−Gv(x,y)

Here, FIGS. 15A to 15D show information of a flaw that appears in acontour image as schematic profiles. In these drawings, FIG. 15A shows aprofile of surface information of the workpiece, FIG. 15B shows aprofile of an inclination image, FIG. 15C shows a profile of a contourimage by means of a forward difference, and FIG. 15D shows a profile ofthe contour image by a central difference. As shown in FIGS. 15A and15C, “(1) forward difference” has an advantage in that a flaw in unitsof one pixel can be clearly seen, but has a disadvantage in that animage displaced by 0.5 pixels from the original image is obtained.

(2. Central Difference)

Next, a method for calculating δ²s/δx² and δ²s/δy² by means of thecentral difference will be described. An inclination image Gh in thehorizontal direction and an inclination image Gv in the verticaldirection are regarded as input, and a pixel G(x,y) at coordinates (x,y)of a contour image E is calculated by the following expression:E(x,y)=Gh(x−1,y)−Gh(x+1,y)+Gv(x,y−1)−Gv(x,y+1)

As shown in FIGS. 15A and 15D, “(2) central difference” has an advantagein that coordinates are not displaced from the original image, but has adisadvantage in that a result slightly blurs.

(2-3. Texture Extraction Image)

Next, a description will be given of a method for removing the surfacestate of the workpiece from an inclination image obtained by thephotometric stereo method, to obtain a texture extraction imagepreferable for detection of a character, and the like. First, textureinformation is calculated from the albedo ρ of the surface of theworkpiece. The albedo ρ is given by the following expression.ρ=|I|/|LSn|

where ρ is an albedo, L is brightness of the illumination, S is a matrixin the illumination direction, n is a normal vector of the surface, andI is a tone value of the image.

It is to be noted that, while it is possible to find one textureextraction image (albedo) by the expression: ρ=|I|/|LSn|, it is alsopossible to find N texture extraction images (albedos) from a normalvector obtained by this expression and N input images (partialillumination images) and synthesize the texture extraction images, so asto find one texture extraction image (albedo). Examples of a specificsynthesis method include an average method and a halation removingmethod.

FIGS. 16A to 16C show examples of the texture extraction image. In thesedrawings, FIG. 16A shows four texture extraction images as input images,FIG. 16B shows a texture extraction image obtained by applying theaverage method to these images, and FIG. 16C shows a texture extractionimage obtained by applying the halation removing method to these images.

(1: Average Method)

The average method is a method where at each pixel, an average value ofN albedos ρ is regarded as a pixel value of that pixel. As shown in FIG.16B, although the shadow is entirely erased, the shadow in a portionwhere halation has occurred in the input image cannot be erased by thephotometric stereo method, and hence an image where an influence of thehalation remains is obtained. That is, in the four input images (partialillumination images) of FIG. 16A, white places are places where halationhas occurred. When averaging is performed by the average method, asshown in FIG. 16B, the roughness is removed to a certain extent and theimage becomes easier to read, but the roughness slightly remains on abase.

(2: Halation Removing Method)

The expression: ρ=|I|/|LSn| itself exceeds its application range due toa limitation on a dynamic range of the camera as the imaging section andthe diversity of reflectivity of the surface of the workpiece, and henceρ includes an error. In order to correct this error, the halationremoving method can be used.

Since a place where halation occurs is decided by a position ofillumination, it is considered that basically, halation does not occurin the same place in the four partial illumination images. Specifically,although halation may occur over two places between two directions, itcan be said that halation basically does not occur in the same placeunless two illuminations are used.

In the halation removing method, at the time of synthesizing aillumination-direction-specific texture extraction image calculated fromN partial illumination images, considering that there is much halationin a partial illumination image with the largest pixel value at eachpixel or in partial illumination images with the largest to N-th largestpixel values, those are removed and the synthesis is then performed.

Specifically, when each of pixels of the fourillumination-direction-specific texture extraction images in the presentembodiment is synthesized with the third largest pixel value (e.g.albedo value or luminance), an image as shown in FIG. 16C is obtained,and an influence of halation can be removed. It is to be noted that,when the fourth largest pixel value is adopted, a slightly dark image isobtained due to an influence of a shadow. On the contrary, when thesecond largest pixel value is adopted, the influence of halationslightly remains.

Further, in the case of the illumination sections being in the eightdirections, the fifth largest pixel value is adopted on the assumptionthat the influence of halation is not exerted on the fifth largest pixelvalue or the following pixel value. According to a test performed by theinventor, it has actually been confirmed that the best image is obtainedwhen the fifth largest pixel value is adopted. Further, it has also beenproved that the influence of the shadow is exerted when the sixthlargest pixel value or the following pixel value is adopted.

It is to be noted that the synthesis method and the averaging are notrestricted to these, and a variety of methods can be used. For example,the foregoing halation removing method and average method may becombined, to sort albedo values and adopt values in a specific ordersfrom the top. For example, the third and fourth values may be averaged.

(Characteristic Size)

Next, a detail of the setting will be described. As described above, atthe time of creating the contour extraction image, the characteristicsize can be set. By setting the characteristic size to not smaller thana predetermined value, a contour extraction image suitable for OCR canbe obtained.

(3-2. Gain)

At the time of creating a contour extraction image or a textureextraction image, in the process of generating each of these images, itis possible to multiple a pixel value of the original image by a gain.

The gain at the time of creating a contour extraction image refers to aconstant at the time of dispersing a pixel value calculated bycalculation processing to a gradation of 0 to 255. For example, when aflaw, a contour or the like is so shallow that it is difficult to graspthe flaw, the contour or the like, a change in gradation of the pixelvalue increases by increasing this gain value, and hence the flaw, thecontour or the like becomes easy to grasp.

Further, the flaw, the contour or the like becomes easy to grasp byperforming adjustment such that, when the pixel value calculated by thecalculation processing exceeds the range of 0 to 255, it is made to bewithin that range, and when the pixel value is smaller than the range of0 to 255, it is extended into that range.

In the foregoing halation removing method, since albedo values aresorted and, for example, the third value from the top is adopted, thebrightness of the generated image cannot be expected. Accordingly, as aresult of removal of regular reflection, the image may become darkcontrary to the expectation. Therefore, in order to adjust thebrightness, the pixel value is multiplied by a predetermined gain at thetime of creating a texture extraction image.

It is to be noted that, also at the time of calculating an inclinationimage, adjustment can be performed by means of a gain such that thepixel value is made to be within the range of 0 to 255.

(3-3. Noise Removing Filter)

At the time of creating an inclination image or the like, calculation isto be performed by a set of simultaneous equations by use of a pluralityof images, but in practice, differential calculation is performed. Here,noise exists in image data which is obtained by imaging by the imagingsection, when the image data is raw data. Therefore, a noise componentmay be emphasized and a contour may become rough at the time of creatingan inclination image. In order to reduce such noise, a noise removingfilter such as a guided filter is used. A general low-pass filter mayhide or remove not only noise but information of a flaw. On thecontrary, a guided filter can remove noise while keeping an edge at thetime of finding the inclination image, which is preferable.

(3-4. Angle-Noise Reduction)

Next, a principle of angle-noise reduction will be described withreference to a schematic view of FIG. 17. As shown in this drawing,there are two illuminations whose incident angles are α and β. When theworkpiece WK is assumed to be formed of the diffusing reflective surfaceand setting is performed such that the inclination of the workpiece WKwith respect to the reference plane is γ, an angle between a lineperpendicular to the reference plane and an incident is θ, thebrightness of reflective light from the illumination α is I_(α) and thebrightness of reflective light from the illumination β is I_(β), γ isgiven by the following expression:γ=arctan(A·|I _(β) −I _(α) |/|I _(β) +I _(α)|), A=cot θ

Angle-noise reduction is to forcibly make the inclination γ be 0 when|I_(β)+I_(α)| is small to a certain extent.

When it is assumed that both I_(β) and I_(α) are extremely dark andI_(β)=2 and I_(α)=1, for example, |I_(β)−I_(α)|/|I_(β)+I_(α)| becomes avalue as large as ⅓. On the other hand when it is assumed that bothI_(β) and I_(α) are bright and I_(β)=300 and I_(α)=200, for example,|I_(β)−I_(α)|/|I_(β)+I_(α)| becomes a value as small as ⅕. I_(β)=2 andI_(α)=1 greatly affects inclination although there is simply apossibility of noise. Thus, in order to reduce an influence of suchnoise, the angle-noise reduction is applied to allow setting of athreshold of |I_(β)+I_(α)| for forcibly making the inclination be 0.

(Structure of Separate Model of Illumination Section and ImagingSection)

In the image inspection using the photometric stereo method,corresponding positions of the illumination section and the imagingsection are required to be strictly defined in advance. For this reason,in the conventional image inspection apparatus, the illumination sectionand the imaging section have been integrally configured. In other words,since the photometric stereo method is a measurement method forperforming accurate three-dimensional measurement after strictlypositioning the relative positions of the imaging section and theillumination section, the degree of freedom in installation positionshas not been originally provided at the time of installing theillumination section and the imaging section. However, in theconfiguration where the illumination section and the imaging section arefixed in advance, the imaging illumination unit with the illuminationsection and the imaging section integrated necessarily becomes large insize, to worsen the handling thereof.

For example, when an obstacle exists in an inspection position, thereoccurs a situation where the obstacle interferes with the imagingillumination unit to prevent installation. In particular, a lens mountedin a camera as one form of the imaging section may have a large size,such as a line camera, a zoom lens or a large-sized macro-lens. When theimaging section becomes longer as in the case of a large-sized lensbeing mounted in the imaging section, it increases the risk ofinterference between an obstacle OS existing around the workpiece WK anda camera e1 or light sources e21 to e24 for illumination arranged aroundthe camera, as shown in FIG. 18. Further, when the camera e1 as theimaging section becomes long, it is considered that illumination lightis partially blocked by the camera e1 or part of the lens as shown inFIG. 19 to disrupt illumination by, for example, casting a shadow on theworkpiece WK. When the imaging illumination unit is arranged to aposition where the imaging illumination unit and the obstacle do notinterfere with each other in order to avoid such a situation as above, adistance between the illumination section and the workpiece (LightWorking Distance: LWD) becomes long, whereby a light amount of theillumination light decreases to lead to deterioration in inspectionaccuracy.

In contrast, if the imaging section and the illumination section can beprovided separately, it is easy to arrange them in positions where theydo not interfere with the obstacle. For example, even when the obstacleOS shown in FIG. 18 exists, the illumination sections 21 to 24 can beinstalled in positions where the obstacle does not interfere, as shownin FIG. 20, by separating the illumination sections 21 to 24 from theimaging section 11. Similarly, even in the case of mounting alarge-sized lens in the imaging section, adjustment can be performed soas to avoid physical interference of the lens 12 to the illuminationsections 21 to 24. By forming a separate model of the illuminationsections 21 to 24 and the imaging section 11 as thus described, it ispossible to adjust them to be in positions where they do not interferewith the obstacle, and the like, so as to enhance the degree of freedomin installation and inspect the workpiece WK in a variety ofenvironments.

Further, it is also possible to adjust an attachment position of theillumination section and the like such that the imaging section does notblock the illumination light. Similarly, it is also possible to adjustan arrangement position of the illumination section so as to suppressinfluences of halation and a shadow. Especially in the photometricstereo method, since it is assumed that the surface of the workpiece isthe diffusing reflective surface, generally, the normal vector of thesurface of an inclination image obtained by the photometric stereomethod is displaced from the normal vector of an actual surface of theworkpiece. Therefore, a countermeasure such as second-orderdifferentiation has been performed. In addition to such acountermeasure, adjustment of the attitude of the illumination sectionis performed so as to avoid the halation at the time of installation ofthe illumination section, which can reduce such displacement. Further,similarly to the mirror surface, such large roughness which causes achange in brightness is adversely influenced by halation. For example,in the case of the workpiece having a cylindrical shape, specularreflection occurs when irradiation is performed with illumination light,but when roughness exists here, it comes to be not a little influencedby the specular reflection. Accordingly, by giving the degree of freedomto the installation position and the angle of illumination light, anadvantage can be obtained in which adjustment is performed in advance soas to reduce such specular reflection and further improve the inspectionaccuracy.

Moreover, in the photometric stereo method, an influence of a shadow isrequired to be considered. In the photometric stereo method, detectionof reflective light is essential for calculating a normal vector fromreflective light of illumination. However, in a case where the workpiecehas a complex shape and is apt to have a shadow, when the illuminationsection is installed in the vicinity of this workpiece, appropriatereflective light is not obtained from a place where light is notreached, which may disrupt calculation of the normal vector. Even insuch a case, the illumination sections 21 to 24 can be installed inoptimum positions at optimum angles by separating the illuminationsections 21 to 24 and the imaging section 11 from each other, so as tosuppress such an influence.

Additionally, a height of illumination light can be changed byseparating the illumination section from the imaging section. As aresult, it is possible to perform inspection where the distance LWD(Light Working Distance) between the workpiece WK and illumination ismade small or, on the contrary, large so as to select appropriatearrangement in accordance with the inspection use.

(1: Case of Making LWD Short)

When the LWD is made short, as shown in FIG. 21A, light from each of theillumination sections 21 to 24 is much applied from a lateral direction.Since halation generally occurs when an incident angle and a reflectionangle are the same, by bringing the illumination sections 21 to 24 andthe workpiece WK close to each other to make the LWD short, a positionwhere halation occurs can be made to be in the vicinity of the outsideof the workpiece WK. In other words, by intentionally making halationapt to occur outside the workpiece WK, halation in the vicinity of thesurface of the workpiece WK can be suppressed. This can increase aneffective visual field of the workpiece WK.

Further, when the LWD is short, illumination is performed at a lowangle. For example, even when there is roughness on the surface of theworkpiece where light of direct illumination from above is diffused andrecognition is difficult, by irradiating the roughness from the obliqueside surface, it is possible to greatly change a contrast even by asmall change in inclination, so as to facilitate grasping a change inshallow roughness. By emphasizing a contrast at a low angle, it ispossible to obtain an advantage that an inclination image and a contourextraction image can be made clear in the photometric stereo method.

On the other hand, a disadvantage in the case of making the LWD short isthat a shadow is apt to occur due to irradiation with illumination lightat a low angle, causing reduction in effective visual field and makingthe photometric stereo processing difficult. Further, since theillumination sections 21 to 24 and the workpiece WK are apt to interferewith each other, making the LWD short is limitedly used for theworkpiece WK with a large height.

(2: Case of Making LWD Long)

On the contrary, when the LWD is made long, as shown in FIG. 21B,irradiation is performed with much illumination light from theillumination sections 21 to 24 from above. This makes a shadow hardlyoccur, thus reducing an invisible region, to improve the accuracy inphotometric stereo processing. Further, since the illumination sections21 to 24 and the workpiece WK hardly interfere with each other, makingthe LWD long can be used preferably with respect to the workpiece WKwith a large height.

On the other hand, since the position where halation occurs is insidethe workpiece WK, the effective visual field may be reduced. Further,the case of the LWD being long corresponds to the case of the LWD beinglong in multi-angle illumination, and a contrast is hardly made, wherebya change in shallow roughness of the surface of the workpiece may beless likely to be grasped. However, when the LWD is long, it is possibleby multi-angle illumination to create a uniform imaging state withlittle reflection and uneven illuminance of the illumination sections 21to 24, so as to obtain an image clearly capturing the surface state ofthe workpiece WK itself.

As thus described, by separating the imaging section and theillumination section, it is possible to obtain an advantage thatinspection is adaptable to a variety of purposes and uses and the degreeof freedom in arrangement can be enhanced. Especially a normal vectorcalculated by the normal vector calculating section has such dependencythat intensity of the normal vector changes in accordance with a changein relative distance between the imaging section and the illuminationsection in an optical axis direction of the imaging section. That is,when a light amount of the illumination section is the same, there is atendency that, the shorter the relative distance, the larger theintensity of the normal vector, and the longer the distance, the smallerthe intensity of the normal vector. The larger the intensity of thenormal vector, the more clearly the inclination (roughness shape) of thesurface of the workpiece appears, and hence the accuracy in the obtainedinspection image also improves. Therefore in the conventionalphotometric stereo method, for improving the accuracy, the relativeposition of the imaging section and the illumination section has beenstrictly defined in advance so as to obtain large intensity of thenormal vector. That is, the intensity of the obtained normal vector hasbeen enhanced by fixing the imaging section and the illumination sectionin advance.

In visual inspection of a flaw on the surface of the workpiece or OCR,high accuracy is not necessarily required, and for example, it may oftenbe sufficient when detection can be performed with the accuracy that candetermine the presence or absence of the flaw or perform OCR on a numberor a character. In such a use, enhancing the degree of freedom ininstallation of the camera and the light is more advantageous for theuser than enhancing the accuracy. Hence in the present embodiment,enhancing intensity of a normal vector is not in focus, but rather thanthat, reduction in intensity of the normal vector is permitted and alevel on which the accuracy necessary for visual inspection issufficiently obtained is kept. In compensation for this, it is madepossible to install a light and a camera which are released fromconstraints of fixation of the illumination section and the imagingsection and which have a higher degree of freedom, leading to a successin improvement in convenience of the user.

In the case of constituting the illumination section and the imagingsection as separate bodies and performing image inspection by thephotometric stereo method, it is necessary to precisely set a relativepositional relation between the illumination section and the imagingsection. If there is a significant difference between a set illuminationdirection and an actual illumination direction, the inspection accuracymay deteriorate or an erroneous result may be outputted. It can beconsidered that, for example, when the user makes a mistake inconnection of a large number of illumination sections or installs themwith azimuths displaced as a whole at the time of installing the imagingsection and the illumination section, accurate inspection cannot beperformed.

For example, there is considered a case where, as shown in FIG. 22, theimaging section is arranged above the workpiece on a vertical line wherethe workpiece is arranged, and further, the illumination sections arerespectively arranged in the north, south, east and west so as tosurround the workpiece. In such arrangement, in the case of acquiring aheight shape of the workpiece by use of the photometric stereo method,when the shapes of each of the illumination sections and theillumination cable are common, mix-up of connection or arrangement canoccur. Specifically, it is configured such that a predeterminedillumination cable is connected to an illumination connector provided inthe illumination dividing unit, the shape of each illumination cable isgenerally made common, and similarly, the shape of each illuminationconnector is common. Accordingly, it can happen that the illuminationcable is connected to a wrong illumination connector. In particular, themix-up is apt to occur when a long illumination cable is drawn. Further,it is can be considered that, when the illumination section is installedabove the periphery of the workpiece, the illumination sections with thesame shape may be mixed up and fixed.

As shown in FIG. 23, also in the example of arranging a ring-likeillumination section, it is necessary to match a direction of theillumination light with a rotational angle of the imaging section. Inthis case, when the imaging section and the illumination section have apoint-symmetric shape such as a cylindrical shape or an annular shape,rotational angles thereof are difficult to determine, and alignment isnot easy. As a result, the rotational position may be displaced from anintended position. In particular, there has been a fact that the userhardly notices the installation error of the illumination sectionbecause a characteristic amount such as a shape of the surface of theworkpiece can be detected to a certain extent even when the rotationalangle, the arranged position or the like is slightly erroneously set.Further, in the case of complete mix-up of the illumination sections,the setting error is relatively noticeable, but in the case of a settingerror like small displacement of a rotational angle, the user hardlynotices the error, leading to deterioration in accuracy, which has beenproblematic.

Accordingly, in the present embodiment, at the time of acquiring partialillumination images obtained by illumination from a plurality ofdifferent illumination directions by the photometric stereo method,there is set an installation auxiliary section for supporting correctsetting at the time of installation such that the illumination directionand the turning-on order of the illumination section are set asintended. Specifically, in FIG. 1, at the time of connecting each of theillumination sections and the illumination controlling section,appropriate connection destinations are shown to the user alongpredetermined installation setting such that the illumination directionmatches with the turning-on order. Alternatively, in order that anattitude (rotational position or rotational angle) of each of theillumination sections in a circumferential direction matches, a correctattitude is instructed to the user at the time of installing theillumination section. Hence it is possible to prevent an error at thetime of installation, so as to accurately perform visual inspection ofthe workpiece by the photometric stereo method.

(Installation Setting)

Here, installation setting is setting in which, in order to correctlycalculate a normal vector by the photometric stereo method, aninstallation position is defined in accordance with the number ofillumination sections so that the workpiece can be irradiated withillumination light from predetermined illumination directions. Forexample, as shown in the plan view of FIG. 2, in the example of usingfour illumination sections, in order to arrange the four illuminationsections of the first illumination section 21, the second illuminationsection 22, the third illumination section 23 and the fourthillumination section 24 around (to the north, south, east and west of)the workpiece, the installation setting includes installation positioninformation for illumination that the first illumination section 21 isarranged in the north, the second illumination section 22 in the east,the third illumination section 23 in the south and the fourthillumination section 24 in the west. Further, the installation settingmay include the turning-on order of the illumination sections and theimaging timing for the imaging section by the illumination controllingsection.

(Installation Auxiliary Section)

Hereinafter, a specific example of the installation auxiliary sectionwill be described. In the example shown in FIG. 1, connectiondestinations of the imaging section 11, the illumination sections 21 to24 and the illumination dividing unit 75 are indicated by theinstallation auxiliary section so as to form an intended connectionwhere the photometric stereo processing is appropriately performed, inother words, such that erroneous connection or arrangement are notperformed. For example, installation written indicators each showing aspecific direction or a rotational angle of arrangement of illuminationare provided in connection portions, or connectors or terminals to beconnected to each other are made to have commonality or correspondenceby means of characters, symbols or other marks, shapes, colors or thelike of installation written indicators, thereby to visually showarrangement positions, directions, rotational angles, connectiondestinations, or the like. In such a manner, by supporting a mutualconnecting operation by the installation auxiliary section, the user canvisually grasp the corresponding relation between the mutual connectiondestinations by referring to the installation written indicators.Hereinafter, FIGS. 24 to 34 show specific examples of the installationauxiliary section.

First Example

FIG. 24 shows a state where a first illumination cable 71, a secondillumination cable 72, a third illumination cable 73 and a fourthillumination cable 74, which are respectively extended from the firstillumination section 21, the second illumination section 22, the thirdillumination section 23 and the fourth illumination section 24 shown inFIG. 2, are connected to the illumination dividing unit. Theillumination dividing unit is provided with an illumination connectorfor connecting each of the illumination cables. By connecting theillumination cable to each of the illumination connectors, electricityis supplied from the illumination controlling section 31 atpredetermined turning-on timing, to allow turning-on of the illuminationsection. In this case, when the illumination cable is erroneouslyconnected to an illumination connector different from the original, apartial illumination image is captured in a different illuminationdirection or at different illumination timing, which results inpreventing correct inspection from being performed. Therefore, byproviding a first installation written indicator showing an illuminationconnector as a connection destination on the illumination connector asthe installation auxiliary section, the illumination connector as theconnection destination is shown to the user, to allow avoidance oferroneous connection. Here, “1” is indicated on the illuminationconnector as a connection destination of the first illumination cable71, thereby allowing the user to visually confirm the connectiondestination of the first illumination section. Further, a secondinstallation written indicator showing information on a connectiondestination is preferably provided also at the connection end of theillumination cable extended from each of the illumination sections. Inthe example of FIG. 24, “1” is indicated as the second installationwritten indicator at the connection end of the first illumination cableextended from the first illumination section. It is thereby possible forthe user to associate “1” on the illumination connector with “1” on thefirst illumination cable and connect them, so as to ensure the accuracyof the connection. In such a manner, by matching the first installationwritten indicator provided on the illumination connector with the secondinstallation written indicator provided on the connection cable, it ispossible to further reduce errors in the connection operation andimprove the reliability of the inspection. Similarly, each of the secondillumination cable 72, the third illumination cable 73 and the fourthillumination cable 74 can be provided with the second installationwritten indicator, and the illumination connectors as connectiondestinations of the cables can also be provided with first installationwritten indicators. As described above, the illumination dividing unitand the illumination controlling section may be integrated.

The installation auxiliary section is also applicable to connectionbetween the illumination cable and the illumination section. That is, asshown in FIG. 25, also in an interface portion that connects the firstillumination cable 71 to the first illumination section, an installationwritten indicator showing the corresponding relation of wiring as theinstallation auxiliary section may be provided. Here, “1” is indicatedas a third installation written indicator on a connector portion of thefirst illumination section to be connected with the first illuminationcable. Further, “1” is also indicated at the end edge of the firstillumination cable. Accordingly, also at the time of the wiringoperation for the first illumination section and the first illuminationcable, mix-up of wiring can be avoided. Especially when a plurality ofillumination sections have the same shape or the illumination cable islong, in other words, even in an environment where a wiring error is aptto occur, it is possible to reliably perform wiring and enhance thereliability of the inspection accuracy. In the example of FIG. 25, onlythe connection between the first illumination section and the firstillumination cable is shown for simplifying the drawing, but it goeswithout saying that the same applies to the second illumination sectionto the fourth illumination section. Further, although each illuminationcable is an insertion type with respect to the illumination section andthe illumination dividing unit in the above example, even in aconfiguration where the illumination cable is directly connected to theillumination section or the illumination dividing unit, the presentembodiment can be applied. Further, although the number has been used asthe installation written indicator in the above example, a visiblewritten indicator such as a letter, a symbol, a mark or a color isapplicable as appropriate. Further, a plurality of written indicatorsmay be combined. For example, when a color and a number is combined toconstitute an installation written indicator, such as 1 in red, 2 inblue, 3 in green and 4 in purple, the wiring relation becomes clearer.Furthermore, the installation written indicator is not restricted to avisible one, but in addition to or in place of this, an installationwritten indicator recognizable by the sense of touch, such as Braille,may be applied. It is preferably usable especially when the connectionportion is minute or printing and marking are physically difficult, orwhen visible confirmation is difficult such as the case of installationin a narrow place or a dark place.

Second Example

In the above example, there has been described the example of supportingthe operation to wire each of the illumination sections to theillumination controlling section by the installation auxiliary section.In this case, the installation auxiliary section is provided on each ofthe illumination sections and the illumination cable. However, theinstallation auxiliary section is not restricted to this, and isapplicable to supporting an operation of installing the illuminationsection at a correct position. In this case, the installation auxiliarysection can be provided on each of the imaging section and theillumination section. Such an example is shown in a schematic plan viewof FIG. 26. In this example, the first illumination section 21, thesecond illumination section 22, the third illumination section 23 andthe fourth illumination section 24, each being formed in an arc shape,are arranged around (here, to the north, south, east and west of) theimaging section in a plan view. As described above, when the shape ofeach of the illumination sections is the same, mix-up can occur at thetime of installing the illumination sections. Therefore, by providing awritten indicator for arrangement of the illumination section as theinstallation auxiliary section on the imaging section, it is possible toperform arrangement in accordance with this written indicator, and avoidmix-up. Here, the top surface of each of the illumination sections iscolored differently, and the same color is also used for a fourthinstallation written indicator on a region of the top surface of theimaging section which corresponds to each of the illumination sections.In this example, the first illumination section 21 is colored in red,the second illumination section 22 is colored in blue, the thirdillumination section 23 is colored in green, and the fourth illuminationsection 24 is colored in purple. This allows the user to distinguish theillumination sections in accordance with the fourth installation writtenindicators provided on the imaging section at the time of theinstallation operation for the illumination sections, and furtherfacilitates the user to visually confirm whether or not the illuminationsections have been correctly installed after the installation. Thus, itis possible to obtain an advantage that a mix-up error of theillumination sections can be prevented.

This is one example, and a color to be used on each of the illuminationsections can be arbitrarily changed. Moreover, the color is notrestrictive, and the discrimination is also possible by means of writtenindicators such as a pattern, a design, a symbol, a number or acharacter.

Further, a position where such a written indicator is provided can bethe side surface or the bottom surface of each of the illuminationsections and the imaging section, other than the top surface thereof.Moreover, the configuration where the installation auxiliary section isprovided on the imaging section is not restrictive, and for example, theinstallation auxiliary section may be provided on another region such asa stage on which workpiece is placed, or a casing for the imageinspection apparatus.

Also in the operation of wiring the illumination cable pulled out fromeach of the illumination sections and the illumination dividing unit, itis possible to indicate correct combination of wiring destinations bythe installation auxiliary section similarly to the above, thereby toprevent an error in the wiring operation and further to facilitateperforming the confirmation operation after the wiring.

Third Example

In the above second example, there has been described the example wherethe relative installation positions at the time of annularly arrangingthree or more illumination sections around the workpiece are confirmedby the installation auxiliary section. The illumination sections canalso be integrally configured in advance as an annular illumination unitas described above. Also in such a case, as shown in FIG. 23, unless theannular illumination unit 20 is fixed by adjusting its relativerotational position, namely rotational angle, with respect to theimaging section, an accurate result of the photometric stereo processingcannot be obtained. Especially when the annular illumination unit 20 hasa point-symmetric shape, a rotational position is not decided, thusmaking the adjustment operation difficult. Even in such a case, therelative rotational positions of the annular illumination unit and theimaging section can be positioned by the installation auxiliary section.For example, as shown in FIG. 27, lines or joints are indicated as theinstallation auxiliary section on boundary portions of the firstillumination section 21, the second illumination section 22, the thirdillumination section 23 and the fourth illumination section 24 whichconstitute the annular illumination unit 20. This clarifies a regionoccupied by each of the illumination sections. Further, this is alsomade to correspond to the imaging section, and lines are indicated onits top surface. Providing such an installation auxiliary sectionfacilitates defining the rotational positions of the imaging section andthe annular illumination unit 20 and further facilitates performingpositioning thereof.

Even when the annular illumination unit 20 is rotated in units of 45°,it cannot be discriminated, and hence a position as a reference isclarified. For example, coloring the first illumination sectionclarifies the attitude of the annular illumination unit 20, and it isthus possible to avoid a circumstance of displacement in units of 45°.Further, a region of the top surface of the imaging section whichcorresponds to the colored first illumination section is preferablycolored in a similar manner. Therefore, matching the colored regionswith each other facilitates determining the rotational angle of theannular illumination unit.

The above is one example, and as for the installation auxiliary section,another written indicator which defines the rotational position of theannular illumination unit can be adopted as appropriate. For example, asin a modified example shown in FIG. 28, a line is indicated in advanceat a position to become a reference of the annular illumination unit 20,and it is defined in advance such that this line is located at a zenith,thereby allowing accurate positioning of even the annular illuminationunit 20 having a point-symmetric shape. Further, as the writtenindicator showing the reference position is not restricted to the line,but an arbitrary pattern such as another mark, a character or a designcan be used. For example, in the example of FIG. 29, a position tobecome a zenith is indicated by an arrow.

The number of reference positions is not restricted to one, and aplurality of reference positions may be provided. For example, as shownin FIG. 30, a different mark may be indicated in a region correspondingto each of the illumination sections. Further, indication of a similarmark to each of these marks on the illumination connector of theillumination dividing unit can also be used to discriminate wiring ofthe illumination cable pulled out from each of the illuminationsections.

The written indicator is not restricted to the mark, and an arbitrarypattern such as a character, a color or a design can be used. Forexample, in the example shown in a plan view of FIG. 31, characters N,E, S and W showing the cardinal directions are indicated. Further,similar characters showing the cardinal directions are also added to theimaging section 11 and the illumination dividing unit 75. This canfacilitate positioning and wiring in accordance with the setting of thephotometric stereo processing.

In the example shown in a plan view of FIG. 32, a colored mark isindicated in a region corresponding to each of the illuminationsections. In this example, the first illumination section 21 is providedwith a red mark, the second illumination section 22 with a blue mark,the third illumination section 23 with a green mark, and the fourthillumination section 24 with a purple mark. Further, providing thesimilar marks to the imaging section 11 and the illumination dividingunit 75 can facilitate performing the positioning and wiring operationsand the confirmation.

Moreover, as the method for determining a reference position, there canalso be adopted a method for eliminating a physical shape by means of apoint-symmetric shape to clarify an attitude from an external shape,other than the configuration where a written indicator showing areference is provided. For example, in the example shown in FIG. 33, itis configured such that a projection having a triangle shape is providedin part of the annular illumination unit 20, and is set at a zenithposition as a reference, thereby to allow positioning. Further, formingsimilar projections in the imaging section 11 and the dividing unit 75allows matching of attitudes and wiring thereof. Moreover, each of theshapes of each of the illumination sections 21 to 24 and theillumination dividing unit 75 is not restricted to a circle in a planview, but an arbitrary shape such as a rectangular shape or a polygonalshape can be used. For example, in the example shown in FIG. 34, arectangular shape is used. Further, it is adequate that the projectioncan serve as a sign showing a position or a direction as a reference andcan employ an arbitrary form such as a depression.

In the example of FIG. 27 described above, the illumination cable ispulled out from the annular illumination unit 20 formed by integratingthe first illumination section 21, the second illumination section 22,the third illumination section 23 and the fourth illumination section24, by being made to correspond to each of the illumination sections,but the illumination cables may be put together in the integratedannular illumination unit 20. This can save labor for the wiringoperation, avoid mix-up, and further simplify the wiring itself. FIGS.28 and 29 show examples of such wiring.

(Image Inspection Method)

A procedure of an image inspection method for performing visualinspection of the workpiece will be described using the image inspectionapparatus 1 with reference to a flowchart of FIG. 35.

In Step ST1, the image processing part 41 issues a trigger signal toeach of the illumination sections 21, 22, 23, 24 via the illuminationcontrolling section 31, and the illumination sections 21, 22, 23, 24 areturned on one by one in accordance with the trigger signal.

In Step ST2, the imaging section 11 is activated every time each of theillumination sections 21, 22, 23, 24 is turned on, to capture an imageof the workpiece WK.

In Step ST3, the imaging section 11 transmits four image signals Q1 toQ4 of the captured images of the workpiece WK to the image processingpart 41.

In Step ST4, by use of the four image signals Q1 to Q4 inputted from theimaging section 11, the image processing part 41 calculates a normalvector of the surface at each pixel with respect to each of the imagesignals Q1 to Q4.

In Step ST5, the image processing part 41 creates reduced inclinationimages respectively obtained by reducing the image into 1/1, 1/2, 1/4,1/8, 1/16 and 1/32, which will be required in post-stage processing,with respect to each of the image signals Q1 to Q4. Further, the imageprocessing part 41 previously creates a texture extraction image, whichwill be required in the post-stage processing, with respect to each ofthe image signals Q1 to Q4.

In Step ST6, a characteristic size is adjusted according to the need. Bychanging the characteristic size, a size of roughness extracted in aroughness extraction image changes. Specifically, when thecharacteristic size is made large, there is obtained a roughnessextraction image where roughness having a large size is extracted. Onthe contrary, when the characteristic size is made small, roughnesshaving a small size is extracted. Therefore, the user adjusts thecharacteristic size in accordance with a size of a flaw to be extracted.Alternatively, in the case of the use for OCR, a roughness extractionimage suitable for OCR can be obtained by increasing the characteristicsize.

In Step ST7, calculation of a roughness extraction image is performed.In this example, the image processing part 41 generates a roughnessextraction image where roughness is extracted in accordance with thecharacteristic size set in Step ST6, and displays the roughnessextraction image on the display section 51.

In Steps ST10 to 12, the image processing part 41 or the PLC 81performs, on a contour extraction image, flaw detection processing fordetecting a flaw by use of a flaw inspection tool, to perform flawdetermination processing for determining whether or not to have detecteda flaw.

First, in Step ST10, the image processing part 41 or the PLC 81specifies a position of an inspection region to become an inspectiontarget with respect to a generated contour image. At the time of settingthe inspection region, an image of the workpiece WK is extracted by, forexample, extracting an edge. When the workpiece WK is not greatlydisplaced, a setting position for the inspection region may beregistered in advance.

In Step ST11, the image processing part 41 or the PLC 81 performs imageprocessing for detecting a flaw in the specified inspection region. Theimage processing part 41 or the PLC 81 calculates a referenceconcentration value by a stored calculation method, and calculates adifference between the reference concentration value and a concentrationvalue of each pixel in the inspection region with respect to each pixel.Then, the image processing part 41 or the PLC 81 executes labelingprocessing (processing of attaching labels with “0, 1, 2, . . . ” togroups of white pixels in a binary image) by means of a set and storedthreshold (a threshold referred to as a flaw amount is decided inadvance), to calculate a characteristic amount with respect to eachspecified flaw. The calculated characteristic amount is, for example,plus/minus information on plus/minus of differences, a total ofdifferences, the maximum value of differences, an average value ofdifferences, or a standard deviation of differences.

In Step ST12, the image processing part 41 or the PLC 81 executes theflaw determination processing on the flaw specified in Step ST11 inaccordance with a determination reference used for flaw determination.When it is determined as a flaw, a place of the flaw is marked on thedisplay section 51, and the processing is completed.

Modified Example

In the above example, there has been described the example where, bymaking a characteristic size adjustable by the user, a roughnessextraction image in which roughness with a size desired by the user isextracted is generated by means of a parameter of the characteristicsize. However, the present invention is not restricted to thisconfiguration, and by preparing a plurality of observation modes inaccordance with the use for observation or a purpose of the user andallowing the user to select an observation mode, it is possible to formsuch a configuration as to generate a desired image. Such an examplewill be described with reference to a flowchart of FIG. 36. Steps ST″1to ST″5 are similar to those in FIG. 35, and hence detailed descriptionsthereof will be omitted.

In Step ST″6, the user is allowed to select an observation mode. In thisexample, any of a contour extraction mode, a texture extraction mode anda roughness mode is made selectable. In each observation mode, acharacteristic size suitable for observation is preset. The user maymanually adjust the characteristic size finely after selecting theobservation mode. The following step differs depending on the selectedobservation mode. That is, the process proceeds to Step ST″7 when thecontour extraction mode is selected, the process proceeds to Step ST″8when the texture extraction mode is selected, and the process proceedsto Step ST″9 when the roughness mode is selected. The process proceedsto Step ST″10 after the processing in each step. In such a manner, theroughness extraction image generating section, the contour imagegenerating section 41 b and the texture extraction image generatingsection 41 c are switchable.

In Step ST″7, the image processing part 41 performs calculation of aroughness extraction image. That is, the image processing part 41displays on the display section 51 a contour extraction image with acharacteristic size for a roughness extraction image to be displayed.

In Step ST″8, the image processing part 41 executes the processing inthe case of the contour extraction mode having been selected by theuser. That is, the image processing part 41 performs calculation of acontour extraction image based on the reduced inclination image createdin Step ST″5, and displays the contour extraction image on the displaysection 51.

In Step ST″9, the image processing part 41 executes the processing inthe case of the texture extraction mode having been selected by theuser. That is, the image processing part 41 performs calculation of acontour extraction image based on the texture extraction image createdin Step ST″5, and displays the texture extraction image on the displaysection 51.

For subsequent Steps ST″10 to ST″12, a similar procedure to that in FIG.35 described above can be used, and detailed descriptions thereof willbe omitted.

According to the above image inspection apparatus, although a primaryinclination image is generated by the photometric stereo method atfirst, a secondary inclination image, namely a contour extraction imageis created by performing differential processing on the generatedprimary inclination image in the X-direction and the Y-direction. Bythis processing, it is possible to reduce a disadvantage of thephotometric stereo method like a conventional one in which the obtainedinspection image greatly changes by slight inclination of illuminationor the installation surface of the workpiece or an error of inputinformation such as an originally inputted illumination position, and aninspection image corresponding to an actual object cannot be obtained,for example as in a phenomenon where a roughness image is obtainedalthough there is actually no roughness, and a phenomenon where an imagein which the center of the workpiece WK is swollen is obtained becauseof the tendency that a place closer to the illumination is normallybrighter. By setting a tone with which a place where a change ininclination is large in a depression direction of the surface becomesdark and setting a tone with which a place where a change in inclinationis small in a projection direction of the surface becomes bright, thereis obtained an image preferable for extracting a flaw, a contour and thelike where inclination of the surface of the workpiece greatly changes.

Further, the contour image generating section creates a plurality ofreduced inclination images with a different reduction ratio from thecalculated normal vector at each of the pixels, performs differentialprocessing in the X-direction and the Y-direction on each of the reducedinclination images, performs weighting such that a reduced contour imagewith a predetermined reduction ratio is greatly reflected to theobtained reduced contour image, to enlarge the image to an originalsize. All the reduced contour images thus enlarged are added to form acontour extraction image. According to the above configuration, althougha size used for flaw determination varies depending on the use of theuser, weighting can be performed such that a reduced contour image witha predetermined reduction ratio is greatly reflected to a contourextraction image. Thus, a contour extraction image where a reducedcontour image with a reduction ratio desired by the user is emphasizedcan be obtained in accordance with the use of the user. As a demand,user may judge a depression over ten pixels as a flaw or may judge adepression over 100 pixels as a flaw. Further, only a steep edge may beextracted as an edge.

Further, a flaw is normally detected over a plurality of frequencies.Since the contour extraction image which is obtained by adding all theenlarged reduced contour images is employed, a flaw, a contour and thelike can be clearly detected without blur as a whole as compared to areduced contour image with only one frequency.

Further, the weighting can be performed by preparing a previouslydecided weighting set, and applying the weighting set to the reducedcontour image, to proportionally divide an adoption ratio of the reducedcontour image with each of the reduction ratios. With thisconfiguration, the weighting set (characteristic size) previouslydecided is preset at the time of synthesis with a contour extractionimage, and hence the user can instantly and easily perform switching toa desired contour extraction image.

The weighting set can include a set that makes large an adoption ratioof a reduced contour image by which a contour extraction image withclear roughness of the surface of the workpiece is obtained. By takingthis way to use as, for example, the roughness mode, it can beseparately used from the contour extraction mode.

Further, since the weighting set can include a set that makes large anadoption ratio of a reduced contour image by which a contour extractionimage suitable for OCR is obtained, it is possible to create an imagepreferable for performing OCR on a carved seal in cast metal, forexample.

Further, there is formed a configuration where, from a calculated normalvector at each of the pixels which exists in number corresponding to thenumber of times of illumination performed by the illumination sections,albedos of each of the pixels in the same number as the number of thenormal vectors is calculated, to generate from the albedos a textureextraction image that shows a design obtained by removing an inclinedstate of the surface of the workpiece, thereby making the contour imagegenerating section and the texture extraction image generating sectionswitchable. Accordingly, although it has been required to search for aplace where a flaw should not exist and decides the place an inspectionregion since it is normally difficult to discriminate an original flawand an originally existing contour, it is possible to perform the flawinspection after performing search in a texture extraction image anddeciding an inspection region. Moreover, also at the time of performingOCR, it is possible to perform OCR after performing search in a textureextraction image and deciding a target region for performing OCR.

The texture extraction image generating section can sort values of thealbedos of each of the pixels in the same number as the number of thenormal vectors, and employs, as the texture extraction image, an imageformed by adopting a pixel value in a specific order from the top.Accordingly, a pixel value with high luminance where halation hasoccurred is not adopted, and a texture extraction image with aninfluence of halation removed therefrom is obtained.

Further, although it is generally common understanding that thephotometric stereo technique is a technique for three-dimensionalmeasurement, by providing a flaw inspection tool required for the flawinspection after generation of a contour extraction image, it ispossible to provide an image inspection apparatus that can be consideredas a practical product obtained by applying the photometric stereotechnique to the flaw inspection.

Further, the image inspection apparatus includes: an imaging section forcapturing an image of a workpiece from a certain direction; anillumination section for illuminating the workpiece from differentdirections at least three times; an illumination controlling section forsequentially turning on the illumination section one by one; an imagegenerating section for driving the imaging section at each illuminationtiming to generate a plurality of partial images; a normal vectorcalculating section for calculating a normal vector with respect to thesurface of the workpiece at each of pixels by use of a pixel value ofeach of pixels having a corresponding relation among the plurality ofimages; and a texture extraction image generating section forcalculating, from a calculated normal vector at each of the pixels whichexists in number corresponding to the number of times of illuminationperformed by the illumination section, albedos of each of the pixels inthe same number as the number of the normal vectors, to generate fromthe albedos a texture extraction image that shows a design obtained byremoving an inclined state of the surface of the workpiece. The textureextraction image generating section can sort values of the albedos ofeach of the pixels in the same number as the number of the normalvectors, and employ, as the texture extraction image, an image formed byadopting a pixel value in a specific order from the top. With thisconfiguration, the texture extraction image generating section sortsvalues of the albedos of each of the pixels in the same number as thenumber of the normal vectors, and employs an image formed by adopting apixel value in a specific order from the top as the texture extractionimage. Accordingly, a pixel value with high luminance where halation hasoccurred is not adopted, and a texture extraction image with aninfluence of halation removed therefrom is obtained.

As thus described, according to an image inspection apparatus of theembodiment, it is possible to inspect a flaw and a printed character ofthe workpiece in an easy and robust manner by the photometric stereomethod.

The image inspection apparatus, the image inspection method, the imageinspection program, and the computer-readable recording medium or therecording device according to the present invention are preferablyusable for an inspection device or a digitalizer using photometricstereo.

What is claimed is:
 1. An image inspection apparatus for performingvisual inspection of a workpiece, the apparatus comprising: three ormore illumination sections for illuminating the workpiece from mutuallydifferent illumination directions; an illumination controlling sectionfor turning on the three or more illumination sections one by one in aturning-on order; an imaging section for capturing an image of theworkpiece from a certain direction at illumination timing for turning oneach of the illumination sections by the illumination controllingsection, to capture a plurality of partial illumination images withdifferent illumination directions; a normal vector calculating sectionfor calculating a normal vector with respect to a surface of theworkpiece at each of pixels by a photometric stereo method by use of apixel value of each of pixels having a corresponding relation among theplurality of partial illumination images captured by the imagingsection; and a contour image generating section for performingdifferential processing in an X-direction and a Y-direction on thenormal vector at each of the pixels calculated by the normal vectorcalculating section, to generate a contour image that shows a contour ofinclination of the surface of the workpiece.
 2. The image inspectionapparatus according to claim 1, wherein the contour image generatingsection creates a plurality of reduced inclination images with adifferent reduction ratio from the calculated normal vector at each ofthe pixels, performs differential processing in the X-direction and theY-direction on each of the reduced inclination images, performsweighting such that a reduced contour image with a predeterminedreduction ratio is reflected to the obtained reduced contour image toenlarge the image to an original size, and adds all the enlarged reducedcontour images, to form a contour extraction image.
 3. The imageinspection apparatus according to claim 2, wherein the weighting isperformed by preparing a previously decided weighting set, and applyingthe weighting set to the reduced contour image, to proportionally dividean adoption ratio of the reduced contour image with each of thereduction ratios.
 4. The image inspection apparatus according to claim3, wherein the weighting set includes a set that makes large an adoptionratio of a reduced contour image by which a contour extraction imagewith clear roughness of the surface of the workpiece is obtained.
 5. Theimage inspection apparatus according to claim 3, wherein the weightingset includes a set that makes large an adoption ratio of a reducedcontour image by which a contour extraction image suitable for OCR isobtained.
 6. The image inspection apparatus according to claim 1,further comprising: an inspection region specifying section forspecifying a position of an inspection region to become an inspectiontarget with respect to the generated contour image; an image processingsection for performing image processing for detecting a flaw within theinspection region specified by the inspection region specifying section;and a determination section for determining presence or absence of aflaw on the surface of the workpiece from result of the processing bythe image processing section.
 7. The image inspection apparatusaccording to claim 1, wherein the imaging section and each of theillumination sections are independent separate members, and are arrangedat arbitrary positions.
 8. The image inspection apparatus according toclaim 1, wherein four illumination sections are provided.
 9. The imageinspection apparatus according to claim 1, wherein the three or moreillumination sections are made up of a plurality of light-emittingelements arranged in an annular shape, and the illumination controllingsection takes a predetermined number of adjacent light-emitting elementsas a first illumination block, simultaneously turns on thelight-emitting elements in the first illumination block, and turns offother light-emitting elements, to make a first illumination sectionfunction as first illumination from a first illumination direction,performs control so as to turn on a second illumination block, which ismade up of a predetermined number of light-emitting elements andadjacent to the first illumination block, to constitute the secondillumination block for performing illumination from a secondillumination direction different from the first illumination direction,and performs control so as to turn on a third illumination block, whichis made up of a predetermined number of light-emitting elements andadjacent to the second illumination block, to constitute the thirdillumination block for performing illumination from a third illuminationdirection different from the first illumination block and the secondillumination direction.
 10. An inspection method for capturing an imageof a workpiece to perform visual inspection, the method comprising thesteps of: illuminating the workpiece from three or more mutuallydifferent illumination directions by illumination sections, andcapturing one partial illumination image with respect to each of theillumination directions by use of a common imaging section whose imagingdirection and relative position with the illumination sections areadjusted in advance, to acquire a plurality of partial illuminationimages with different illumination directions; calculating a normalvector with respect to the surface of the workpiece at each of pixels bya normal vector calculating section by use of a pixel value of each ofpixels having a corresponding relation among the plurality of partialillumination images with different illumination directions; performingdifferential processing in an X-direction and a Y-direction on thecalculated normal vector at each of the pixels, to generate a contourimage that shows a contour of inclination of the surface of theworkpiece by a contour image generating section; and performing visualinspection of the workpiece by use of the contour image.
 11. An imageinspection program embedded in a non-transitory computer readable mediumfor capturing an image of a workpiece to perform visual inspection,wherein the program allows a computer to realize functions of;illuminating the workpiece from three or more mutually differentillumination directions by illumination sections, and capturing onepartial illumination image with respect to each of the illuminationdirections by use of a common imaging section whose imaging directionand relative position with the illumination sections are adjusted inadvance, to acquire a plurality of partial illumination images withdifferent illumination directions; calculating a normal vector withrespect to the surface of the workpiece at each of pixels by a normalvector calculating section by use of a pixel value of each of pixelshaving a corresponding relation among the plurality of partialillumination images with different illumination directions; performingdifferential processing in an X-direction and a Y-direction on thecalculated normal vector at each of the pixels, to generate a contourimage that shows a contour of inclination of the surface of theworkpiece by a contour image generating section; and performing visualinspection of the workpiece by use of the contour image.
 12. Anon-transitory computer-readable recording medium, in which the imageinspection program according to claim 11 is recorded.